Table of Contents

• 1  Convolutional Neural Networks (CNN, ConvNet)
  • 1.1  Why CNNs?
  • 1.2  Convolution Operation
  • 1.3  Example with edges filters
  • 1.4  Padding
  • 1.5  Strided Convolutions
  • 1.6  Convolutions on volumes (RGB images)
  • 1.7  Layers in ConvNets
  • 1.8  Schematic Description of a ConvNet
  • 1.9  CNNs in Tensorflow
• 2  Hands-on
  • 2.1  steps
  • 2.2  Dropout
  • 2.3  Augmentation
  • 2.4  Transfer Learning

Convolutional Neural Networks basics

and an example with classification

(LENS ML School 2021; Emmanouela Rantsiou)

Convolutional Neural Networks (CNN, ConvNet)

• a class of artificial Neural Networks
• widely used in computer vision*
• learn hierarchy of features

*Computer vision is an essential, complex, wide-spread, and ever developing part of AI. Computer vision tasks include:

• Image Classification
• Classification with localization
• Object detection
• Image captioning
• action classification
• Semantic segmentation
• Instance segmentation
• Neural Style Transfer

Why CNNs?

• Need not to be restricted to small input images.

• Image of 1300 X 1300px --> Input on a layer of a FC (Dense) network with e.g. 1000 hidden units $\rightarrow$ parameters (weights) to calculate : (1000,1300x1300) = 1.69E9 parameters (x3 for a color image) $\rightarrow$ Computationally expensive and data hungry.

Convolution Operation

Example with edges filters

(edge detectors)

image = PIL.Image.open('brick2.jpeg').convert('L')
brick = asarray(image)
plt.figure(figsize=(20,20))
plt.imshow(brick,cmap='gray')
plt.axis('off')
plt.show()

image = plt.imread('filters_conv.jpeg')
plt.figure(figsize = (20,20))
plt.imshow(image)
plt.axis('off')
plt.show()

out_conv_h=signal.convolve2d(brick,filter_h,mode='valid')
out_conv_v=signal.convolve2d(brick,filter_v,mode='valid')
f, axarr = plt.subplots(1,2,figsize = (50,50))
axarr[0].imshow(np.absolute(out_conv_h),cmap='gray')
axarr[0].set_axis_off()
axarr[1].imshow(np.absolute(out_conv_v),cmap='gray')
axarr[1].set_axis_off()

plt.show()

image = PIL.Image.open('brick.jpeg')
brick = asarray(image)
plt.figure(figsize=(10,10))
plt.imshow(brick)
plt.axis('off')
plt.show()

out_corr_v=signal.correlate(brick,filter_v_3d,mode='valid')
out_corr_h=signal.correlate(brick,filter_h_3d,mode='valid')

f, axarr = plt.subplots(1,2,figsize = (50,50))
axarr[0].imshow(np.absolute(out_corr_h),cmap='seismic')
axarr[0].set_axis_off()
axarr[1].imshow(np.absolute(out_corr_v),cmap='seismic')
axarr[1].set_axis_off()
plt.show()

Padding

Strided Convolutions

Convolutions on volumes (RGB images)

Layers in ConvNets

The building blocks (i.e. most commonly used types of layers in CNNs)

• Convolutional layers
• Pooling layers
  • Max Pooling
  • Average Pooling
  • Global Pooling
• Dense (Fully Connected) layers

Schematic Description of a ConvNet

CNNs in Tensorflow

import tensorflow
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras import models

model = models.Sequential()
model.add(layers.Conv2D(8, (3, 3), activation='relu', input_shape=(80,80,3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(16, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(64,activation='relu'))
model.add(layers.Dense(4, activation='softmax'))

Hands-on

• Simple classification example using a ConvNet

Classification/Regression

• Regression:
  • output is a prediction on a quantity which takes continuous values
• Classification:
  • output is a prediction on a class, i.e. discrete labels
    • binary classification
    • multi-class classification
      • single-label classification
      • multi-label classification

steps

1. Data
   • set of 8 different classes of images: airplane, car, cat, dog, flower, fruit, motorbike, person
   • create the relevant datasets
2. CNN model
   • build the CNN model, train it with the given data, check performance
   • we will observe overfitting
3. Improve previous CNN model: i.e. avoid overfitting
   • Dropout (interfere in the model architecture)
   • Augmentation (interfere with the data)
4. Transfer learning
   • Do not use own CNN model, but borrow a pretrained one. Use it on our given data.

Dropout

Dropout: A Simple Way to Prevent Neural Networks from Overfitting, JMLR (2014) http://jmlr.org/papers/v15/srivastava14a.html

Augmentation

Augmenting the training dataset, by applying transformations to existing data (images in this case): flip, rotate, shear, shift, zoom, color distortion, etc.

Transfer Learning